Leveraging AI to support managers with Insights for improving employee performance appraisal
It was an ordinary Schlumberger, passionate, discussion about business in Q2 of 2019 with a colleague in operations that triggered events leading to the deployment of the Performance Appraisal Advisor Report to managers in December 2019. He was asking whether People Analytics is using the performance management data to build some trends and insights back to management as he read an article in HBR on the importance of appraisal in driving results for the company. To validate his intuition, we ran a quick survey targeting 500 people in various functions, geographies and seniority levels. While the results were expected from a talent management view as mentioned by Fabrice Bouchaud, our HR director - WorkForce planning & talent management, it was still an astonishing majority, given the diversity of our company across different cultures, business units, and others.
With the launch of PeopleFirst, we expanded the survey to about 15,000 employees globally. The responses indicated that we (managers) could improve in providing feedback and performance appraisal to employees where ~40% were not satisfied with the most recent one conducted. We also noticed a correlation between manager and employee dissatisfaction, in what we coined as the cascading effecting, that is if the manager was not satisfied with their manager’s feedback in their own performance appraisal, then their employees are more than likely to be the same.
We pursued the idea and launched a long, text mining, exercise for a period of two months analyzing 10 years’ worth of employee performance appraisal data to build a product that will support our managers. The performance appraisal report was prepared using well established machine-learning, text-mining, and sentiment analysis techniques, including BERT, NLTK, and TextBlob, while normalizing them to Schlumberger context through statistical techniques. The results were normalized to provide company specific benchmarks:
The length of appraisal and effective use of text is used to gauge the time and effort invested. When managers’ comments are left blank or cut short, it is deemed a missed opportunity.
An objective appraisal is based on facts, by opposition to opinions. The analysis here detects whether subjective inputs like references to gender, cultural settings, age, health, and physical attributes are used.
Further use of sentiment analysis is quantifying concepts like positive reinforcement in the text. Emphasis on achievements and encouragement to build on these are proven to drive excellence.
Our talent management team (owner of performance management process) was working on developing some new training material for our managers and guideline that will help them which was to be integrated in the product.
A social comparison nudge approach was chosen to deliver the insights back to managers in a private manner, while including employee level data for them to enhance their understanding of how the tool work. The approach was selected after extensive research on use cases to improve sales performance, manufacturing productivity, and others in the people analytics space.
The early results of the deployment were positive. With over 8,000 managers viewing the report since it was deployed, we have seen an overall improvement in management feedback of more than 30% based on the use of text and more than 25% increase in positive re-enforcement to their employees. More than 500 managers wrote back to us (in surveys and direct emails) with positive feedback as well as recommendations for improvements. The report gained a Net Promoter Score of 78 (on scale of -100 to +100) on usefulness and value.
While plenty of work remains to improve the product based on the feedback and the long-term impact is yet to be assessed, the team was excited to see that it had a positive impact given a short development cycle. It was one of the early projects we took on in the People Analytics team that established the potential of analytics in HR space.
I’m constantly looking after new technologies and see how they can blend into sub-surface modeling workflows. ML (Statistical Learning), Elastic Cloud-Compute, Quantum Algorithms are amongst emerging tools that I keep a close eye on with some interesting prototype results… but this is for another post…
Author information: Modhar Khan is the People Analytics manager with our human resources function. His job as “a data driven—Sherlock Holmes” is to find business use cases that analytics could support and bring true value to, while collaborating and engaging leaders across the function. He spends the rest of his time as a data evangelist raising awareness over the potential value that AI and machine learning could bring to the company to support business performance, employee experience, diversity and inclusion.
